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Abstract 

Background: Control of reaching movements for manual work, vehicle operation, or interactions with manual 
interfaces requires concurrent gaze control for visual guidance of the hand. We hypothesize that reaching 
movements are based on negotiated strategies to resolve possible conflicting demands placed on body segments 
shared by the visual (gaze) and manual (hand) control systems. Further, we hypothesize that a multiplicity of 
possible spatial configurations (redundancy) in a movement system enables a resolution of conflicting demands 
that does not require sacrificing the goals of the two systems. 

Methods: The simultaneous control of manual reach and gaze during seated reaching movements was simulated 
by solving an inverse kinematics model wherein joint trajectories were estimated from a set of recorded hand and 
head movements. A secondary objective function, termed negotiation function, was introduced to describe a means 
for the manual reach and gaze directing systems to balance independent goals against (possibly competing) 
demands for shared resources, namely the torso movement. For both systems, the trade-off may be resolved 
without sacrificing goal achievement by taking advantage of redundant degrees of freedom. Estimated joint 
trajectories were then compared to joint movement recordings from ten participants. Joint angles were 
predicted with and without the negotiation function in place, and model accuracy was determined using the 
root-mean-square errors (RMSEs) and differences between estimated and recorded joint angles. 

Results: The prediction accuracy was generally improved when negotiation was included: the negotiated 
control reduced RMSE by 16% and 30% on average when compared to the systems with only manual or visual 
control, respectively. Furthermore, the RMSE in the negotiated control system tended to improve with torso 
movement amplitude. 

Conclusions: The proposed model describes how multiple systems cooperate to perform goal-directed human 
movements when those movements draw upon shared resources. Allocation of shared resources can be undertaken 
by a negotiation process that is aware of redundancies and the existence of multiple solutions within the individual 
systems. 
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Background 

Many daily activities include hand reaching movements 
devised to bring the hand to a desired target. For accurate 
hand reaches, it is crucial to capture visual images of the 
environment and self (body parts). Visual information 
provides feedback for accurate guidance of movements 
[1,2] and plays an important role in the generation of 
movement plans as well as the calibration of movement 
systems with respect to the environment [3-5]. 

These perspectives suggest that reaching movements 
involve the coordination of two effector systems: the 
hand being guided to the target and the gaze being di- 
rected at the target. Simultaneously achieving accurate 
reaching and appropriately directing gaze is not a trivial 
task, as the central nervous system (CNS) has to control 
both the manual and visual systems, each of which relies 
on different sets of body segments and end-effectors. 
The CNS must solve complex problems from multiple 
layers of reference frame conversions [6], spatial trans- 
formation [7] and sensory-motor integration through 
feedback and feed-forward control [8] . 

Furthermore, the actions of these systems are not 
completely independent from one another, as they share 
common body segments (links). Both the arms/hands 
and neck/head are connected to the torso, thus the actions 
of one system often impose disturbances on the other. 
That is, hand movements may be accompanied by torso 
movements, which would also influence head and eye 
positions. The torso is in effect a resource shared by the 
manual and visual systems, and demands made on this 
shared resource may compete. 

Hence in planning and executing visually guided hand 
reaching movements, the allocation of shared resources 
becomes critical to the manual system (MS) and visual 
system (VS). When demands on shared resources are in 
conflict, a rule must be established in a manner that 
allows these systems to compromise their demands on 
the shared resources. Thus it is hypothesized that the 
CNS employs a negotiation system (NS), which considers 
the demands from the MS and VS and allocates the 
contribution of the torso in a manner that serves the 
goals of both the MS and VS. 

The negotiation between multiple systems is necessary 
because of the existence of shared resources (torso 
movement) and negotiation with resolution of conflict 
(benefiting both the MS and VS) is made possible by 
the existence of redundant degrees of freedom in both 
the MS and VS. Redundancy implies the existence of 
multiple spatial configurations of each linkage system 
that achieve the same hand reach position (or gaze 
direction in the case of the VS). By taking advantage of 
redundancy, conflicting demands on torso movement 
can be resolved, while satisfying the desired end effector 
movements. For example, the MS may require torso 



flexion while the VS requires torso extension. In such 
cases, both systems will have to surrender a portion of 
torso movement, but by virtue of redundancy, they 
may increase the utilization of the uniquely dedicated 
remaining body segments so that the aims of hand and 
gaze are both best fulfilled. 

Given the complex multi-link system of the human 
body, the problem to solve is the determination of joint 
angle sets that closely mimic real movement realizations. 
When the available degrees of freedom (the number of 
joints in the body) is greater than the dimension of 
the task space (the number of coordinates required to 
describe the end-effector configuration), the solution 
space may be very large and the relationships between 
joint space variables and task space variables can be 
highly nonlinear. In robotics, a commonly used method 
for solving the inverse kinematics problem (determining 
joint angles given end-effector coordinates) is known 
as differential inverse kinematics. Differential inverse 
kinematics relies on inversion of the relationship be- 
tween joint angular velocities and end-effector veloci- 
ties, or inversion of kinematics in the velocity domain. 
The differentiated kinematics is linear in the joint and 
end-effector velocities, simplifying the process of in- 
version (solving for the joint velocities in terms of the 
end-effector velocities) (see [9] for a review). In the case of 
redundancy, where more than one feasible solution is 
possible, optimization is commonly used to identify the 
best solution given a pre-defined objective function. 
Such objective functions can be defined to minimize the 
(possibly weighted) sum of joint velocities [10-12], distance 
to mechanical joint limits [13], or distance to obstacles [14]. 
For our goal of modeling the coordination of two systems 
(the MS and VS) with shared body segments, however, 
an additional objective function is needed to implement 
negotiation. 

Thus the primary goal of the framework developed 
here is to estimate the joint angles that place the end- 
effectors of the MS and VS in the desired locations. 
The secondary goal is to allocate the use of shared 
resources (shared body segments or links) for both 
systems according to a negotiation scheme. To validate 
this framework, human movements in three-dimensional 
seated right-hand reaching tasks were simulated with and 
without the negotiation scheme. These simulations took 
the form of solutions of the inverse kinematics of the 
redundant systems, as driven by recorded hand and 
head movements only. The benefit of the negotiation 
framework was then quantified by comparing simulated 
and recorded movements, in terms of the movements 
of all body segments. It is hypothesized that negoti- 
ation is a mechanism adopted by the CNS to control 
multi-segmental movements of the human body and 
therefore provides better predictions of body movements. 
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This framework would also help to understand the in- 
teractions between the manual and visual systems in 
goal-directed movements. 

Methods 

Movement modeling 

Seven revolute joints corresponding to the hand, fore- 
arm, upper arm and torso comprise the manual system 
variables, which can be represented by an angle vector 

9m = [$ml> @m2> @m3> @m4> 0m5> $m6> ^rnA ■ All joint 

variables were represented in the Denavit-Hartenberg 
convention [15,16], as listed in Table 1. Similarly, the 
visual system configuration 0 V = [0„i, 0 v2 > 0v3> 0v4> 0v5> 
8 v6 ] T is described using six revolute joints correspond- 
ing to the head, neck, and torso. Since three joints of 
the torso (corresponding to flexion/extension, axial ro- 
tation, and lateral bending at the L5/S1 joint) are 
shared by both systems, the corresponding variables 
can be represented by common terms 9\, d 2 , and 0 3 
and the above definitions can be rewritten as 0 m = 

[01, 0 2 , 03) 0m4) 0m5» 9 m 6> ® mlA an d Q v = [01, d 2 , 6 3 , 

0v4. 0v5> d v6 ] T , respectively. Also by conjoining 0 m and 

0 V , 0 = [01, 02, 03, 0 m 4> 0m5> 0m6> 0jk7i 0v4> 0y5> ®v(\ ■ 

With the MS, upper arm axial rotation (0 m6 ) is equiva- 
lent to internal/external rotation at the shoulder. Wrist 
joints (3 additional degrees of freedom: flexion/exten- 
sion, pronation/supination, and ulnar/radial deviations) 
were not considered, to similarly match the VS joint 
count. 

The position of the right hand p m , which is the manual 
system end-effector, is described by Cartesian coordinates 



global 



global 



global 



as in p m \ Phand (it)' Phand (yY Phand (z) 



mK Q m ) where 

global global j global j / ... . i , 

Phand («). Phand (y) and Phand (z) denote*- (positive right- 
ward), y- (positive forward), and z- (positive upward) coor- 
dinates of the hand in a global coordinate system, with the 
origin coinciding with the L5/S1 joint position. The symbol 
f m denotes a function of the direct (forward) kinematics of 
hand reaching movements describing hand position as a 
function of joint angles. 



The end-effector position of the visual system is defined 

^ Pv = [Pt7gi W . PtaTgi W . (J" =/v(«v). in Which 

Parget {x y P target [y) and Ptargtt (z) represent the position 
of the target along the x-, y-, and z-axes in a head-centered 
reference frame. The symbol f v denotes the direct kine- 
matics function for the visual system. 

The goal of the present model is to estimate a joint 
angle vector (0) based on the recorded end-effector 
position using the inverse kinematics function f~ m (p m ), 
or /y J (p v ). Using the differential kinematics method, 
the velocity p of the end-effector position p can be ob- 
tained using 



dp dp de 3/(8), , 
p = dt = de dt = ^ e = mQ 



(i) 



where J is a Jacobian matrix that is a function of the 
given joint configuration 0. Equation 1 can be adapted 
to represent either the manual system p m = J m 0 or 
visual system p v = J F 0, where J m and J v represent the 
Jacobian matrices for the manual and visual system, 
respectively. The solution of inverse kinematics can 
then be obtained by solving for the joint angular 
velocities 0 m and 0 V and integrating with feedback 
stabilization [9,17]. 

Due to the existence of redundant degrees of freedom 
(non-square matrix J), the inverse of J does not exist. 
However, a weighted pseudo-inverse (denoted J f ) may be 
used in which the squared sum of all joint velocities is 
minimized [10,11]. Thus from Equation 1, 



0 = W- 1 J T (JW- 1 J T ) -I p = fp 



(2) 



where W is a weighting matrix that characterizes the 
instantaneous contribution of each joint. In this study, 
W is a diagonal matrix whose entries are the peak 
magnitudes of joint velocities in recorded individual 
movements. 

A secondary objective function implements the ne- 
gotiation scheme {negotiation function) . This function 



Table 1 Link segment compositions for the manual and visual system 




Manual system 






Visual system 


Variable 


Joint movement 


Positive rotation 


Variable 


Joint movement 


6ml 


Torso flexion/extension 


Extension 




Torso flexion/extension 


6m2 


Torso axial rotation 


Counter-clockwise 




Torso axial rotation 


6m3 


Torso lateral bending 


Leftward 


6k3 


Torso lateral bending 


6m4 


Shoulder flexion/extension 


Flexion 


6„4 


Neck/Head flexion/extension 


6m5 


Shoulder abduction/adduction 


Abduction 




Neck/Head axial rotation 


6m6 


Upper arm axial rotation 


Counter-clockwise 


Si* 


Neck/Head lateral bending 


6m7 


Elbow flexion/extension 


Flexion 
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reconfigures the joint angles of the link system without 
changing the end-effector position by utilizing the re- 
dundant degrees of freedom. The projection matrix 
(I— J 1- J) 0o, where I represents the identity matrix, can 
be used to project an arbitrary vector 6o into the null 
space of J. Thus from Equation 2, the manual system 
equation can be adapted as follows. 

o=r m p m +(i-rj m )9 0 (3) 

As described in [14,17], multiplying Equation 3 by J„ 
would result in J„0 = ] v T m P m + J v (l-J^J v )8 0 . Solving for 
8 0 would thus result in 0 O = [j v (l-0 m )] (] v Q-] v f m p m ). 
By substituting for 0o from Equation 3: 

» = V m P m + [Ul-Tjm)V(W-l£jm) ( 4 ) 

Since (l-T m J m ) is Hermitian and idempotent, Equation 4 
can be simplified as [14]: 

e = r m p m + [h(i-rj m )]\p v -hV m p m ) (5) 

Then 0 can be obtained from the numerical integra- 
tion of 6 and the cumulative error of the end-effector 
position predictions is reduced by a feedback control 
algorithm [14]. 

In this study, recorded movements (described below) 
provided end-effector velocity p m and p v , and initial 
posture angles used for input parameters. Since the ul- 
timate goal of this model is to compare torso move- 
ments predicted by the MS, VS, and NS respectively, 
the torso joint variables were included as a part of each 
system and each system was designed to generate joint 
movement estimations as follow: First, the manual and 
visual systems were separately considered to compute the 
corresponding joint angle estimations (0 m and 8„. respect- 
ively) using Equation 3, with 9 0 set to a null vector. 
Subsequently Equation 5 was employed to construct 
the negotiation system and compute the corresponding 

joint angle estimations ^6„^. 

Movement recording 
Participants 

Five males and five females with a mean age of 22.3 ± 
1.8 years (mean±SD) participated in movement record- 
ing as paid volunteers. All participants were right-handed, 
free from any known musculoskeletal or neurological 
disorders, and had normal vision (20/20 or better) 
without corrective lenses. Mean stature and body weight 
were 170.9 cm (SD: ± 12.0) and 67.2 kg (SD: ± 16.3), 
respectively. 



Equipment 

Visual targets were placed on an arc (radius = 115 cm, 
arc length = 300 cm) set horizontally in front of the 
participant (Figure 1). The arc position was adjusted so 
that the mid-point of the arc coincided with the mid- 
sagittal plane of each participant. The elevation of the 
arc was set either at eye level or 50 cm below eye level 
(denoted as at-eye and below-eye level, respectively). 
The horizontal forward distance between the arc and 
the participant's sternum was either 100% (Figure 1A) or 
155% (Figure IB) of the individual hand reach distance. 
Reach distance was defined as the length between the 
right acromion process and the tip of the right index 
finger while the upper arm, forearm, and hand were ex- 
tended horizontally at shoulder level. The mean reach 
distance across all participants was 65 cm. 

Each target was composed of alphanumeric characters 
(0 - 9, A, C, E, F, H, L, U, or P) displayed on a seven- 
segment LED whose visual angle was approximately 0.5°. 
Four targets were placed in each of the left and right 
hemispheres. In the 100% reach distance condition, 
the interval between the targets was approximately 
15°, and the leftmost and rightmost target positions 
were approximately ± 60° of azimuth with respect 
to the mid-sagittal plane, respectively. In the 155% reach 
distance condition, the target interval was approximately 
10° and the most eccentric positions corresponded 
to ± 40°. A separate LED display, used as the initial 
fixation point, was placed in the mid-sagittal plane at eye 
level for all arc elevation settings. 

The participant was seated on a chair (seat pan 
height = 40 cm, seat pan width = 50 cm, back support 
height = 58 cm). A pad equipped with a micro switch 
was placed on the participant's right lap and served as 
the home position of the right index finger. The position 
of the lap switch was adjusted so that the elbow-included 
angle was approximately 90° when the index finger was 
placed on the button. The entire room was dimly lit 
during the experiment. 

Movement recording 

An electromagnetic motion capture system (Flock of 
Birds'", Ascension Technology) with five sensors placed 
on the forehead, upper torso (C7), lower torso (L5), 
upper arm, and right hand, was used to record the 
movements of the head, neck, clavicle, torso, upper arm, 
forearm and hand. A splint was fastened with an elastic 
band to the palmar side of the index finger to maintain 
its posture constant throughout the experiments so that 
the coordinates of the six degree of freedom sensor on 
the right hand could be used to estimate the fingertip 
position while imposing minimal interference on the 
reach task. The Cartesian coordinates and orientations 
of the sensors were used to estimate the joint center 
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1 00% Reach Distance 



At Eye Level 
50cm 
Below Eye Level 




Figure 1 Target configurations. A: 100% reach distance. B: 155% reach distance. 



locations [18,19]: for example, the sensor placed on the 
upper arm provides the position of the shoulder and 
elbow using the measured upper arm length. Similarly, 
clavicle movements were estimated from the sensor at 
the upper torso and computed position of the shoulder 
joint. The sensor movements were sampled at 25 Hz, 
and trajectories of each landmark were smoothed off-line 
by a second order Butterworth low-pass filter with a 6 Hz 
cutoff frequency. Since the electromagnetic motion cap- 
ture system is sensitive to electromagnetic interference, 
testing equipment including the target arc and seat were 
made of non-metallic materials such as wood and com- 
posites. Linearity within the volume of measure was 
verified during the calibration procedure. 

Procedures 

When the initial fixation point was illuminated, accom- 
panied by a 500 Hz signal tone with a 0.1 s duration, the 
participant was asked to align the nasion with the initial 
fixation point and depress the switch on the right lap 
pad with the right index finger. This defined the resting 
initial posture. After a delay of 2 seconds, the initial 
fixation display was turned off and a target was displayed 
at a randomly selected eccentricity. A 2000 Hz tone with a 
0.1 s duration signaled the participant to initiate the reach 
movement. The participant was asked to point just below 
the target with the right index finger, which activated 
the micro switch placed on the fingertip to signal the 
completion of the hand movement. The alphanumeric 
characters displayed on the active target were changed 
once per second. The participant was asked to read each 
alphanumeric character aloud throughout the trials to 
ensure that gaze was maintained on target. The target 
was turned off 2 seconds after fingertip contact. 



Each block of target presentations was composed 
of sixteen trials (8 target locations x 2 replications) 
for a given target arc distance and elevation. A total 
of four blocks (2 distances x 2 arc elevations) were 
recorded for each participant. A three-minute break 
was provided at the completion of each block. The order 
of block presentation was balanced and randomized 
across participants. 

The procedures were reviewed and approved by the 
University of Michigan Health Sciences Institutional 
Review Board, and all participants signed an informed 
consent prior to the experiment. 

Data processing 

Movement onset was determined by the start of either 
hand or head movement, whichever occurred first. The 
hand movement onset was determined by the activation 
of the microswitch. The head movement onset was esti- 
mated by off-line analysis. Specifically, the start of a head 
movement was defined as the time when the head had 
been stationary for the previous 120 ms (3 consecutive 
sampling frames) and engaged in active rotation for the 
next 120 ms. The threshold for active head rotation was 
defined as an angular velocity > 257s. The completion of 
movement was determined by the fingertip microswitch 
contact with a target zone. 

Body segment angles were calculated from the joint 
center positions estimated from sensor positions [18,19]. 
For the torso, for example, the vectors from the L5/S1 
to the C7/T1 and right shoulder joint were used to 
construct a rotation matrix describing torso orientation, 
from which the associated three joint angles {d\. flexion/ 
extension, f9 2 : axial rotation, and 63: lateral bending) were 
calculated as 
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[ cost?! cosfj? 3 + sinf^ sin# 2 sin# 3 - sinf^ cos# 2 
Rforeo = cos6?3 sinf5i - cosc?i sinf?2 sinf?3 cosf?i cosf3 2 
- cos(9 2 sinf3 3 - sinf?2 



cos0! sin# 3 - cos0 3 sin^ sin# 2 
sinf3i sinf93 + cosf3i cosf33 sinf32 
cosf3 2 cosf9 3 



In a similar way, the shoulder angles were calculated 
from the shoulder-to-elbow and hand-to-elbow vectors, 
and neck/head angles from the nasion-to-C7Tl and 
right-to-left tragion vectors. All joint angles were defined 
as listed in Table 1. 

Model performance analyses 
Model accuracy evaluation 

The joint angles estimated by the MS, VS and NS were 
compared with recorded movements in order to evaluate 
the accuracy performance of each model. Specifically, the 
joint angle error at movement completion (end-posture 
errors) and root mean square error (RMSE) throughout 
the movement duration were computed with respect to 
the corresponding recorded joint movements. RMSE was 
computed by 



and their respective ratio to the sum of difference mag- 
nitudes was estimated using: 



RMSE : 



E (&'M-ftM 



where 6i(t) and 6i{t) denote the estimated and recorded 
angle of the i-th joint at the t-th. measurement sample, 
with a total number of measurement samples n for the 
given trial. Errors from the NS model were compared 
with the MS and VS model errors. All joints including 
the torso and upper extremity in the MS and the torso 
and neck/head in the VS (Table 1) were considered and 
compared with the corresponding joint in the NS, 
respectively. Statistical significance was tested using 
bootstrap confidence intervals, which measures the 
empirical distribution of the parameters of interest by 
re-sampling the original observations. Bootstraping is 
known to be useful when common parametric statistical 
analysis may not be employed due to errors not normally 
distributed [20-22]. 

Allocation of torso joint contribution 

It was hypothesized that the NS weighs the demands for 
the torso joint use from the MS and VS, then allocates 
torso contribution for each system depending on the 
assessed weight. Thus if the prediction by the NS is 
closer to the prediction by the MS than the prediction 
by the VS, it was interpreted that the NS has allocated 
larger torso joint contribution to the MS than the VS, or 
vice versa. Specifically, the magnitudes of differences in 
torso joint angles were compared between the NS and 
MS estimations and between the NS and VS estimations, 



A, 





6m~ 


9 Ni 




0Mr 


■0n,\\ 


9v t 





(6) 



where A Mi denotes the proportion of torso movement 
contribution allocated to the MS for the i-th joint angle, 
dM n 9v i and 6 m, denote the i-th joint angle at movement 
completion estimated by the MS, VS, and NS, respectively. 
From its definition, A M i approaches 1 as the NS esti- 
mation approaches MS estimation. Conversely, A Mi 
approaches 0 as the NS estimation approaches the VS 
estimation. A M i was computed for three torso joint an- 
gles for each trial and averaged for different targets, 
grouped by distances (100 vs. 155% reach distance), el- 
evations (at eye level vs. below eye level), and azimuths 
(two targets at the leftmost, middle, and rightmost posi- 
tions). Data processing and modeling work were performed 
using Matlab 2009 (Mathworks, Natick, MA). 

Results 

Overall accuracy estimation 

The RMSEs were overall improved when the NS was 
taken into account in the model, when compared to the 
models with the MS and VS alone (Table 2). Specifically, 
the NS showed smaller RMSEs than the MS (p < 0.05) 
for all torso joints (flexion/extension, axial rotation, 
and lateral bending) and elbow flexion/extension joint 
(p < 0.05). The RMSEs were also smaller for the shoulder 
joint angles but statistical significance was not reached. 
Similarly, when compared to the model with the VS, 
the NS model showed smaller RMSE (p < 0.05) for all 
neck joints (flexion/extension, axial rotation and lateral 
bending). 

The decreased RMSE for the NS was estimated by the 
ratios of (RMSE MS -RMSE NS )/RMSE MS and (RMSE VS - 
RMSE NS )/RMSE VS , which indicated that the NS reduces 
RMSE by 16% and 30% on average when compared to 
the MS and VS, respectively. The improvements in RMSE 
are most prominent for torso flexion/extension (41%) 
when compared to the MS and neck axial rotation (48%) 
when compared to the VS, respectively. 

Similar results were found for the error magnitudes at 
movement completion (Table 3). Errors for torso (flexion/ 
extension, axial rotation, and lateral bending), shoulder 
(flexion/extension), and elbow (flexion/extension) joints 
were significantly smaller (p < 0.05) for the NS than the 
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Table 2 RMSE in joint angle estimations 







MS 






VS 






NS 




NS error 
reduction from MS 


Mean 


SE 


95% CI 


Mean 


SE 


95% CI 


Mean 


SE 


95% CI 


9, 


2.43 


0.08 


[2.27, 2.58] 


1.88 


0.08 


[1 .72, 2.03] 


1.44 


0.06 


[1.33, 1.55] 


*41% 


e 2 


5.49 


0.15 


[5.19, 5.79] 


6.30 


0.18 


[5.94, 6.66] 


4.13 


0.11 


[3.93, 4.34] 


*25% 


e 3 


3.14 


0.11 


[2.92, 3.35] 


3.43 


0.13 


[3.17, 3.70] 


2.39 


0.10 


[2.20, 2.58] 


*24% 


9 m 4 


10.98 


0.33 


[10.33, 11.63] 








10.39 


0.32 


[9.75, 1 1 .02] 


5% 


6m5 


13.45 


0.34 


[12.77, 14.12] 








13.12 


0.35 


[12.43, 13.80] 


2% 


6m6 


10.99 


0.30 


[10.40, 11.58] 








10.92 


0.31 


[10.32, 11.52] 


1% 


6 m 7 


7.88 


0.20 


[7.50, 8.27] 








7.04 


0.17 


[6.71, 7.37] 


*11% 


6v4 








1.43 


0.06 


[1.32, 1.54] 


1.27 


0.06 


[1.16, 1.38] 




6v5 








6.25 


0.22 


[5.83, 6.68] 


3.23 


0.09 


[3.06, 3.40] 




6v6 








6.21 


0.16 


[5.88, 6.53] 


3.96 


0.11 


[3.74,4.18] 





MS: Manual system. VS: Visual system. NS: Negotiation system. SE: Standard error. 95% CI: 95 percentile confidence interval, ^indicates a statistical significance 
at p < 0.05. 



MS model. Similarly, The NS showed smaller errors than 
the VS (p < 0.05) for all torso and neck joints. On average, 
the error magnitudes are 15 and 39% less for the NS than 
the MS and VS, respectively. The largest error reduction 
by the NS was found in torso (35%) and neck (60%) axial 
rotations with respect to the MS and VS, respectively. 

Torso movement profiles 

Since the distinctive role of the NS is to control the 
shared (torso) joints based on the MS and VS demands, 
torso movement profiles were investigated more in detail 
in this and the following sections. For all torso joints 
including flexion/extension, axial rotation and lateral 
bending, the model with the NS generally showed the 
closest estimation to the recorded movement when com- 
pared to the models with either the MS or VS alone. For 
example, in torso movement profiles averaged across all 
participants reaching for the leftmost target at the 100% 



reach distance at eye level, the flexion/extension angle in 
the recorded movement (RM) was -8°(flexion) at the end 
of the movement (Figure 2A). While the VS and MS esti- 
mated -3° and -4° flexion, the NS estimated a larger flexion 
(-6°), which is the closest to the recorded movement. 

In the same condition, the recorded movement showed 
leftward axial rotation increasing to 18° (Figure 2B). When 
model estimations were compared, all models (MS, VS, 
and NS) showed leftward axial rotations in agreement 
with the recorded movement. However, when com- 
pared to recorded movements, the MS shows an error 
"overshooting" up to 26°, while the VS showed an error 
"undershooting" to 8°. With the NS, the estimation was 
a slight overshoot to 21° which is in-between the MS 
and VS predictions. Accordingly, the NS provides esti- 
mation closest to the recorded movement. A similar 
tendency was observed from lateral bending angles 
(Figure 2C). 



Table 3 End posture errors in joint angle estimations 







MS 






VS 






NS 




NS Error 
Reduction 
from MS 


Mean 


SE 


95% CI 


Mean 


SE 


95% CI 


Mean 


SE 


95% CI 


9, 


2.69 


0.1 


[2.49, 2.88] 


2.94 


0.13 


[2.68, 3.19] 


2.01 


0.09 


[1.84,2.19] 


*25% 


e 2 


7.99 


0.25 


[7.49, 8.48] 


8.65 


0.27 


[8.13, 9.17] 


5.19 


0.16 


[4.88, 5.51] 


*35% 


e 3 


4.48 


0.18 


[4.13, 4.83] 


5.54 


0.22 


[5.12, 5.96] 


3.48 


0.16 


[3.18, 3.79] 


*22% 


9m4 


14.72 


0.48 


[13.78, 15.66] 








13.68 


0.47 


[12.76, 14.60] 


*7% 


6m5 


16.63 


0.52 


[15.61, 17.64] 








16.67 


0.55 


[1 5.59, 1 7.74] 


0% 


6m6 


13.92 


0.44 


[13.07, 14.78] 








13.34 


0.44 


[12.47, 14.21] 


4% 


6m7 


8.67 


0.27 


[8.14, 9.20] 








7.80 


0.25 


[7.32, 8.28] 


*10% 


6 v4 








1.80 


0.08 


[1.65, 1.95] 


1.47 


0.08 


[1 .32, 1 .62] 




6v5 








9.17 


0.33 


[8.52, 9.82] 


3.68 


0.13 


[3.42, 3.93] 




6v6 








8.97 


0.26 


[8.47, 9.47] 


4.63 


0.15 


[4.34, 4.92] 





MS: Manual system. VS: Visual system. NS: Negotiation system. SE: Standard error. 95% CI: 95 percentile confidence interval, ^indicates a statistical significance 
at p < 0.05. 



Kim et al. Journal of NeuroEngineering and Rehabilitation 201 4, 11:1 02 
http://www.jneuroengrehab.eom/content/1 1/1/102 



Page 8 of 12 




Allocation of torso joint contribution 
Torso flexion/extension joint 

The torso joint contribution for each system allocated 
by the NS was estimated using Equation 6. The overall 
mean allocation ratio to the MS (all targets) is 0.47 
(SE: ±0.02), which indicates that the allocations of torso 
contributions to the MS and VS are approximately similar. 
However, the MS allocation ratio was significantly larger 
(p < 0.05) for the 155% reach distance (0.52 ± 0.02) than 
the 100% reach distance (0.43 ± 0.03), as illustrated in 
Figure 3. Similarly, targets on the left side are associated 
with a larger [p < 0.05) allocation to the MS (0.68 ± 0.04) 
than those in the mid-sagittal plane (0.38 ± 0.04) or on 
the right side (0.36 ± 0.04), as shown in Figure 3. However, 
target elevation (either below or at eye level) does not 
significantly influence the allocation. Similarly, neither 
target distance x elevation nor target distance x azimuth 
interaction showed significant effects. However target 
elevation x azimuth shows a significant interaction effect 
(p < 0.05). Pairwise comparisons using Bonferroni correc- 
tions indicate that on the right side, the allocation ratio to 
the MS is greater for targets at eye level than targets below 
eye level (p < 0.05). 

Torso axial rotation joint 

The overall mean allocation ratio to the MS for axial rota- 
tion was 0.60 (SE 0.02) which indicates that a significantly 



larger (p < 0.05) allocation was made to the MS than the 
VS. Neither target distance nor elevation induces statisti- 
cally significant effects (Figure 4). However, target azimuth 
was a significant factor (p < 0.05). The MS allocation 
ratio was smaller (p < 0.05) for targets on the right side 
(0.40 ± 0.03) than for targets in the mid-sagittal plane 
(0.70 ± 0.02) or on the left side (0.69 ± 0.02). A significant 
distance x elevation interaction effect (p < 0.05) indicates 
that allocation to the MS is larger for 155% than 100% 
reach distance for targets below eye level, while it is com- 
parable for those at eye level. Similarly, larger allocations 
are made to the MS for targets at 155% than 100% reach 
distance on the right side, but they are comparable for 
targets on the left side and in the mid-sagittal plane 
(distance x azimuth interaction, p < 0.05). However, 
the elevation x azimuth interaction was not statistically 
significant. 

Torso lateral bending joint 

The overall mean allocation ratio to the MS for lateral 
bending was 0.57 + 0.02, which indicates that a signifi- 
cantly larger (p < 0.05) allocation is made to the MS than 
the VS (Figure 5). Both reach distance and target azimuth 
induce statistically significant differences (p < 0.05). Specif- 
ically, the 155% reach distance induces a larger (0.64 ± 0.03) 
MS allocation ratio than the 100% reach distance (0.50 ± 
0.02). For different target azimuths, pairwise comparisons 
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Figure 3 Proportion of torso movement contribution allocated to the MS (torso flexion/extension). Values on the y-axis are in a [0,1] scale 
and unitiess (as defined in Equation 6). A value > 0.5 indicates that a larger torso movement contribution is allocated to the MS than the VS. 'denotes 
a statistically significant difference at p < 0.05. A: Distance, B: Elevation, and C: Azimuth effects. 



indicate that MS allocations are smaller for targets on the 
right side (0.40 ±0.04) than targets in the mid-sagittal 
plane (0.67 ± 0.02) and on the left side (0.71 ± 0.04). The 
distance x azimuth interaction is also significant ip < 0.05), 
as MS allocations are smaller for the 100% than 155% 
reach distance, and the corresponding difference becomes 
more predominant for targets on the right side than in the 
mid-sagittal plane or on the left side (p < 0.05). Also 
elevation x azimuth interaction indicates that targets 
below eye level are associated with a smaller MS allocation 
ratio than targets at eye level on the left side (p < 0.05), 



while differences are diminished for mid-sagittal plane 
and right side targets. 

Torso movement amplitude and RMSE improvements 

Estimation errors were also quantified as a function of 
movement amplitudes for different torso joints (Figure 6). 
The torso movement amplitude was defined as the angu- 
lar range (=max - min angle) of a joint in a recorded 
movement. Error improvement was defined as the de- 
creased RMSE when the NS was employed in the model 
(=RMSE M s _ RMSE NS )- The error improvements were 
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Figure 4 Proportion of torso movement contribution allocated to the MS (torso axial rotation). Values on the y-axis are 
unitiess (as defined in Equation 6). A value > 0.5 indicates that a larger torso movement contribution is allocated to MS than VS. 
statistically significant difference at p < 0.05. A: Distance, B: Elevation, and C: Azimuth effects. 
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Figure 5 Proportion of torso movement contribution allocated to MS (torso lateral bending). Values on they-axis are in a [0,1] scale and 
unitless (as defined in Equation 6). A value > 0.5 indicates that a larger torso movement contribution is allocated to MS than VS. *denotes a 
statistically significant difference at p < 0.05. A: Distance, B: Elevation, and C: Azimuth effects. 



r ■> 



25°/ 


50% 


75% 




100% 








A. 





0 10 20 30 40 50 60 70 



25% 


50% 


75% 




100% 








B. 





0 5 10 15 20 25 30 35 40 



25 


4 50%| 


75% 




100% 










C. 



0 10 20 30 40 50 60 70 



Recorded Movement Amplitude (°) 

Figure 6 Error improvements by the NS as a function of recorded 
torso movement amplitudes. The RMSE improvement by NS was 
defined as RMSE MS -RMSE NS . Recorded torso movement amplitude was 
defined as max - min angle. A: Flexion/extension, B: Axial rotation, and 
C Lateral bending. Error bars indicate 95th percentile confidence 
intervals. 25%, 50%, 75%, and 100% denote 25 th , 50 th , 75* and 100 th 
percentile torso movement amplitudes, respectively. 



averaged for each of the incremental intervals correspond- 
ing to 0-25 th , 25-50 th , 50-75 th , and 75- 100 th percentile of 
torso movement amplitudes. 

In general, RMSE improves as torso movement amp- 
litude increases. For example, for the flexion/extension 
joint (Figure 6A), the 75-100 th percentile amplitude inter- 
val, which corresponds to approximately 33-67° flexion, 
showed that the NS reduces RMSE by 2° on average. 
However, in the 0-25 th percentile amplitude interval 
(0-4° flexion), the RMSE improvement is 0.4° on average. 
Overall, Pearson's correlation coefficient (R) is 0.35 
(p < 0.05). Similar tendencies were observed for axial 
rotation (Figure 6B; R = 0.29, p < 0.05) and lateral bending 
(Figure 6C; R = 0.27, p < 0.05). 

Discussion 

We hypothesized that during visually guided reaching 
movements the MS and VS make different, sometimes 
incompatible demands for torso movements. We modeled 
a means by which competing demands are resolved using 
a negotiation framework in the differential inverse 
kinematics. This framework accounted for the ability 
of both the MS and VS to compromise on torso move- 
ments without sacrificing hand or gaze movements by 
virtue of MS and VS redundancy. The negotiation 
function allocated torso movements after apportioning 
null space movements in both the MS and VS. 

The results indicated that the predictive power of the 
estimated joint angle trajectories is improved when 
the negotiation framework is implemented. RMSE and 
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end-posture estimation accuracy improved with the NS 
compared to models based on either the MS or VS only. 
These results suggest that upper body movements are 
more accurately described with a system incorporating 
both the manual reaching and visual gaze transition com- 
ponents simultaneously, as opposed to considering only 
one component over another exclusively. 

The NS generates torso movement profiles similar to a 
weighted average of the MS and VS estimations (Figure 2). 
In comparison with the MS or VS, the NS estimations are 
closer to the recorded movements. However, estimation 
errors are not always smaller from the NS than from the 
MS or VS. In several cases, either the MS or VS estimates 
are closer to the recorded movements. These apparent 
discrepancies may result from the effective contribution of 
the torso, since the estimation errors decrease when torso 
movement amplitude increases (Figure 6). In other words, 
the benefit of the NS is greater for movements requiring 
larger torso displacement amplitudes, which underlines 
the necessity to reach a compromise through negotiation 
when torso movements must satisfy the combination of 
two systems demands. 

Thus the results of this study suggest that the CNS, in 
planning and organizing upper body movements, weighs 
the demands from each movement system and allocates 
the shared resources, i.e., the torso and other upper body 
joint movements, in accordance with weighted demands. 
The NS seems to have allocated different amounts of 
torso contribution to the MS and VS depending on target 
location (Figure 3, 4 and 5). Different target locations are 
associated with significantly different hand-to-target 
distances. For example, targets on the left side corres- 
pond to longer hand-to-target distances than those on 
the right side (69 cm versus 51 cm on average respectively, 
for targets below eye level at 155% reach distance), since 
reaching movements were performed with the right hand. 
The longer hand-to-target distance requires torso move- 
ment toward the target to extend the hand reaching 
distance. Indeed torso contribution to the MS is larger 
than the VS for targets on the left side, and vice versa 
for targets on the right side. From a biomechanical 
perspective this contribution is a necessity well predicted 
by the NS. 

In the same way that torso movements vary with 
biomechanical demands, upper body posture can be sig- 
nificantly influenced by the demands for visual image 
acquisition [23,24]. In this regard, we may assume that 
targets in the mid-sagittal plane at eye level impose 
minimal demands for head and torso movement to orient 
gaze, while eccentric targets away from the mid-sagittal 
plane and/or eye level require more contribution from the 
head, neck, and torso. The decreased allocation of torso 
contribution to the MS (thus increased allocation to the 
VS) in the elevation x azimuth effects (Figures 3, 4 and 5) 



show consistent findings. This hypothesis is also in agree- 
ment with our previous results [18], obtained in a similar 
seated condition, which showed that torso axial rotation 
begins to appear for visual targets with azimuth angles 
greater than 60°. 

The negotiation function may be viewed as an integra- 
tion of a number of cost functions contributing to body 
segment coordination. It may be presumed that disruption 
of coordination by neurological disorders such as stroke 
or Parkinson's disease affecting CNS functions may be as- 
sociated with a disruption of the hypothesized negotiation 
process. Hence, the proposed model may find application 
in the analysis of neurological disorders. 

In comparison with other differential inverse kinematics 
models of upper body movement [10,11], which have ig- 
nored visual constraints, the novelty of this study resides in 
a biomechanical system model allocating resources to vis- 
ual information acquisition. The limitation of the current 
study includes the absence of eye movement recordings, 
which could have helped quantify VS behaviors more 
precisely. Given the fast speed of eye movements, how- 
ever, it is posited that gaze lands on target before the hand 
or in the very early phase of hand reaching movement [7] . 
Furthermore, the participants were to read aloud the char- 
acters sequentially displayed on each target. This require- 
ment effectively constrained the gaze to remain on target 
throughout the reaching movement. Hence it is assumed 
that eye orientation (eye in head) is equal to the angular 
offset between the head and target. This allows us to indir- 
ectly estimate gaze orientation using head and target posi- 
tions. Further, the head and neck were modeled as a single 
link rotating about the C7/T1 joint, instead of separate 
links. However, with the complex musculature and coup- 
ling of cervical spine movements, neck movements are 
generally constrained by head movements [25,26] . 

It should be also noted that the proposed model as- 
sumes that the goals of both the MS and VS are pursued 
simultaneously at any given time throughout a reaching 
movement. However as the number of systems and goals 
the CNS needs to consider increases (for example, if 
walking, balance control, bimanual reach, etc. were to be 
additionally included), the simultaneous control of multiple 
systems, as outlined here, may no longer be an efficient 
solution. The RMSE still remaining in the NS system is 
potentially related to such additional goals, not only those 
for fulfilling visual and manual demands. Alternatively, 
the actions of the controlled systems may be divided and 
sequenced. Such a division has been suggested from the 
observation of multiple phases in joint angles and end- 
effector trajectories [2,27,28]. 

Conclusions 

Overall, the proposed model suggests that goal-directed 
human movements are performed by multiple cognitive 
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and sensorimotor systems, each of which is in charge of 
different aspects of the overall movement goal. These 
goals are pursued using body segments either shared by 
both systems or dedicated uniquely to each system. Thus 
one of the crucial functions of the CNS is to allocate 
shared system resources and coordinate the functioning 
of multiple systems in a manner that maximally satisfies 
(and thereby resolves) the competing demands on shared 
resources. Hence coordination can be viewed as a process 
of negotiation. 
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